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Abstract 

In this paper, we systematically investigate both the synfire propagation and firing rate propagation in feedforward 
neuronal network coupled in an all-to-all fashion. In contrast to most earlier work, where only reliable synaptic 
connections are considered, we mainly examine the effects of unreliable synapses on both types of neural activity 
propagation in this work. We first study networks composed of purely excitatory neurons. Our results show that both 
the successful transmission probability and excitatory synaptic strength largely influence the propagation of these two 
types of neural activities, and better tuning of these synaptic parameters makes the considered network support stable 
signal propagation. It is also found that noise has signiflcant but different impacts on these two types of propagation. 
The additive Gaussian white noise has the tendency to reduce the precision of the synfire activity, whereas noise with 
appropriate intensity can enhance the performance of firing rate propagation. Further simulations indicate that the 
propagation dynamics of the considered neuronal network is not simply determined by the average amount of received 
neurotransmitter for each neuron in a time instant, but also largely influenced by the stochastic effect of neurotransmitter 
release. Second, we compare our results with those obtained in corresponding feedforward neuronal networks connected 
with reliable synapses but in a random coupling fashion. We confirm that some differences can be observed in these 
two different feedforward neuronal network models. Finally, we study the signal propagation in feedforward neuronal 
networks consisting of both excitatory and inhibitory neurons, and demonstrate that inhibition also plays an important 
role in signal propagation in the considered networks. 
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1 Introduction 



A major challenge in neuroscience is to understand how the neural activities are propagated through different brain 
regions, since many cognitive tasks are believed to involve this process (Vogels and Abbott, 2005). The feedforward 
neuronal network is the most used model in investigating this issue, because it is simple enough yet can explain propagation 
activities observed in experiments. In recent years, two different modes of neural activity propagation have been intensively 
studied. It has been found that both the synchronous spike packet (synfire), and the firing rate, can be transmitted across 
deeply layered networks (Abeles 1991; Aertsen et al. 1996; Diesmann et al. 1999; Dicsmann ct al. 2001; Cateau and 
Fukai 2001; Gewaltig et al. 2001; Tetzlaff et al. 2002; Tctzlaff ct al. 2003; van Rossum et al. 2002; Vogels and Abbott 
2005; Wang et al. 2006; Aviel et al. 2003; Kumar et al. 2008; Kumar et al. 2010; Shinozaki et al. 2007; Shinozaki et al. 
2010). Although these two propagation modes are quite different, the previous results demonstrated that a single network 
with different system parameters can support stable and robust signal propagation in both of the two modes, for example, 
they can be bridged by the background noise and synaptic strength (van Rossum et al. 2002; Masuda and Aihara 2002; 
Masuda and Aihara 2003). 

Neurons and synapses are fundamental components of the brain. By sensing outside signals, neurons continually 
fire discrete electrical signals known as action potentials or so-called spikes, and then transmit them to postsynaptic 
neurons through synapses (Dayan and Abbott 2001). The spike generating mechanism of cortical neurons is generally 
highly reliable. However, many studies have shown that the communication between neurons is, by contrast, more or 
less unreliable (Abeles 1991; Raastad et al. 1992; Smetters and Zador 1996). Theoretically, the synaptic unreliability 
can be explained by the phenomenon of probabilistic transmitter release (Branco and Staras 2009; Katz 1966; Katz 1969; 
Trommershauser et al. 1999), i.e., synapses release neurotransmitter in a stochastic fashion, which has been confirmed 
by well-designed biological experiments (Allen and Stevens 1994). In most cases, the transmission failure rate at a given 
synapse tends to exceed the fraction of successful transmission (Roscnmund et al. 1993; Stevens and Wang 1995). In 
some special cases, the synaptic transmission failure rate can be as high as 0.9 or even higher (Allen and Stevens 1994). 
Further computational studies have revealed that the unreliability of synaptic transmission might be a part of information 
processing of the brain and possibly has functional roles in neural computation. For instance, it has been reported 
that the unreliable synapses provide a useful mechanism for reliable analog computation in space-rate coding (Maass 
and Natschlager 2000); and it has been found that suitable synaptic successful transmission probability can improve the 
information transmission efficiency of synapses (Goldman 2004) and can filter the redundancy information by removing 
autocorrelations in spike trains (Goldman et al. 2002). Furthermore, it has also been demonstrated that unreliable 
synapses largely influence both the emergence and dynamical behaviors of clusters in an all-to-all pulse-coupled neuronal 
network, and can make the whole network relax to clusters of identical size (Friedrich and Kinzcl 2009). 
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Although the signal propagation in multilayered feedforward neuronal networks has been extensively studied, to the 
best of our knowledge the effects of unreliable synapses on the propagation of neural activity have not been widely discussed 
and the relevant questions still remain unclear (but see the footnote-'^). In this paper, we address these questions and 
provide insights by computational modeling. For this purpose, we examine both the synfire propagation and firing rate 
propagation in feedforward neuronal networks. We mainly investigate the signal propagation in feedforward neuronal 
networks composed of purely excitatory neurons connected with unreliable synapses in an all-to-all coupling fashion 
(abbr. URE feedforward neuronal network) in this work. We also compare our results with the corresponding feedforward 
neuronal networks (we will clarify the meaning of "corresponding" later) composed of purely excitatory neurons connected 
with reliable synapses in a random coupling fashion (abbr. RRE feedforward neuronal network). Moreover, we study 
feedforward neuronal networks consisting of both excitatory and inhibitory neurons connected with unreliable synapses in 
an all-to-all coupling fashion (abbr. UREI feedforward neuronal network). 

The rest of this paper is organized as follows. The network architecture, neuron model, and synapse model used in 
this paper are described in Sec. 2. Besides these, the measures to evaluate the performance of synfire propagation and 
firing rate propagation, as well as the numerical simulation method are also introduced in this section. The main results 
of the present work are presented in Sec. 3. Finally, a detailed conclusion and discussion of our work are given in Sec. 4. 

2 Model and method 

2.1 Network architecture 

In this subsection, we introduce the network topology used in this paper. Here we only describe how to construct the URE 
feedforward neuronal network. The methods about how to build the corresponding RRE feedforward neuronal network 
and the UREI feedforward neuronal network will be briefly given in Sees. 3.4 and 3.5, respectively. The architecture of the 
URE feedforward neuronal network is schematically shown in Figure 1. The network totally contains L = 10 layers, and 
each layer is composed of Ng = 100 excitatory neurons. Since neurons in the first layer are responsible for receiving and 
encoding the external input signal, we therefore call this layer sensory layer and neurons in this layer are called sensory 
neurons. In contrast, the function of neurons in the other layers is to propagate neural activities. Based on this reason, 
we call these layers transmission layers and the corresponding neurons cortical neurons. Because the considered neuronal 
network is purely feedforward, there is no feedback connection from neurons in downstream layers to neurons in upstream 
layers, and there is also no connection among neurons within the same layer. For simplicity, we call the i-th neuron in the 

^An anonymous reviewer kindly reminded us that there might be a relevant abstract (Trommershauser and Diesmann 2001) discussing the 
effect of synaptic variability on the synchronization dynamics in feedforward cortical neural networks, but the abstract itself docs not contain 
the results presumably presented on the poster and also the follow-up publications do not exist. 
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Figure 1: Network architecture of the URE feedforward neuronal network. The network totally contains 10 layers. The 
first layer is the sensory layer and the others are the transmission layers. Each layer consists of 100 excitatory neurons. 
For clarity, only 6 neurons are shown in each layer. 

j-th layer neuron (i, j) in the following. 
2.2 Neuron model 

We now introduce the neuron model used in the present work. Each cortical neuron is modeled by using the integrate- 
and-fire (IF) model (Nordlie et al. 2009), which is a minimal spiking neuron model to mimic the action potential firing 
dynamics of biological neurons. The subthreshold dynamics of a single IF neuron obeys the following differential equation: 



dt 



T,n^=V,,st-V,j+RI,j, (2.1) 



with the total input current 



7,^ ^ I^p + (2.2) 

Here i = 1,2, . . . , Ng and j = 2,3, . . . , L, Vij represents the membrane potential of neuron r„i = 20 ms is the 

membrane time constant, T^ost ~ ^60 mV is the resting membrane potential, i? = 20 Mfi denotes the membrane 
resistance, and /^^J" is the total synaptic current. The noise current = ^/2Dt^ij (t) represents the external or intrinsic 

fiuctuations of the neuron, where (t) is a Gaussian white noise with zero mean and unit variance, and Dt is referred to 
as the noise intensity of the cortical neurons. In this work, a deterministic threshold-reset mechanism is implemented for 
spike generation. Whenever the membrane potential of a neuron reaches a fixed threshold at Vth = — 50 mV, the neuron 
fires a spike, and then the membrane potential is reset according to the resting potential, where it remains clamped for a 
5-ms refractory period. 



On the other hand, we use different models to simulate the sensory neurons depending on different tasks. To study 
the synfirc propagation, we assume that each sensory neuron is a simple spike generator, and control their firing behaviors 
by ourselves. While studying the firing rate propagation, the sensory neuron is modeled by using the IF neuron model 
with the same expression (see Eq. (2.1)) and the same parameter settings as those for cortical neurons. For each sensory 
neuron, the total input current is given by 

la=I{t)+Ifr'', (2.3) 

where i = 1,2, Ng index neurons. The noise current has the same form as that for cortical neurons but with the 

noise intensity Dg. I(t) is a time- varying external input current which is injected to all sensory neurons. For each run of 
the simulation, the external input current is constructed by the following process. Let ry(t) denote an Ornstcin-Uhlenbeck 
process, which is described by 

rc^ = -v{t) + V2Am, (2-4) 

where ^(t) is a Gaussian white noise with zero mean and unit variance, Tc is a correlation time constant, and A is a 
diffusion coefficient. The external input current I{t) is defined as 

\v(t) ilv(t)>0, 

lit) = <^ (2.5) 
[o iii]{t)<0. 

Parameter A can be used to denote the intensity of the external input signal I{t). In this work, we choose A ~ 200 
nA^ and Tc = 80 ms. By its definition, the external input current I(t) corresponds to a Gaussian-distributed white noise 
low-pass filtered at 80 ms and half-wave rectified. It should be noted that this type of external input current is widely 
used in the literature, in particular in the research papers which study the firing rate propagation (van Rossum et al. 
2002; Vogels and Abbott 2005; Wang and Zhou 2009). 

2.3 Synapse model 

The synaptic interactions between neurons are implemented by using the modified conductance-based model. Our modeling 
methodology is inspired by the phenomenon of probabilistic transmitter release of the real biological synapses. Here we 
only introduce the model of unreliable excitatory synapses, because the propagation of neural activity is mainly examined 
in URE feedforward neuronal networks in this work. The methods about how to model reliable excitatory synapses and 
unreliable inhibitory synapses will be briefly introduced in Sees. 3.4 and 3.5, respectively. 

The total synaptic current onto neuron is the linear sum of the currents from all incoming synapses, 

= E ^i'^r, k,j ~ 1) • (E^yn - V,j). (2.6) 
fc=i 
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In this equation, the outer sum runs over ah synapses onto this particular neuron, G{i,j;k,j — 1) is the conductance 
from neuron {k,j — 1) to neuron and £^syn = mV is the reversal potential of the excitatory synapse. Whenever 

the neuron {k,j — 1) emits a spike, an increment is assigned to the corresponding synaptic conductances according to the 
synaptic reliability parameter, which process is given by 

G(»,j;fc,j- 1) ^ G{i,j;k,j-l) + J{i,j;k,j-l) ■ h{i, j; k, j ^ 1), (2.7) 

where h(i,j; k,j — 1) denotes the synaptic reliability parameter of the synapse from neuron (k^j — 1) to neuron j), and 
J{i,j;k,j — 1) stands for the relative peak conductance of this particular excitatory synapse which is used to determine 
its strength. For simplicity, we assume that J{i,j; k,j — 1) = g, that is, the synaptic strength is identical for all excitatory 
connections. Parameter p is defined as the successful transmission probability of spikes. When a presynaptic neuron 
{k,j — 1) fires a spike, we let the corresponding synaptic reliability variables h(i,j;k,j — 1) = 1 with probability p and 
h(i,j;k,j — 1) = with probability 1 — p. That is to say, whether the nem'otransmitter is successfully released or not is 
in essence controlled by a Bernoulli on-off process in the present work. In other time, the synaptic conductance decays by 
an exponential law: 

-^G(z,j;fc,j-l) = lG(z,j;fc,j-l), (2.8) 
at Ts 

with a fixed synaptic time constant t^. In the case of synfire propagation, we choose = 2 ms. and in the case of firing 
rate propagation, we choose Ts — 5 ms. 

2.4 Measures of the synfire and firing rate propagation 

We now introduce several useful measures used to quantitatively evaluate the performance of the two different propagation 
modes: the synfire mode and firing rate mode. The propagation of synfire activity is measured by the survival rate and the 
standard deviation of the spiking times of the synfire packet (Gewaltig et al. 2001). Let us first introduce how to calculate 
the survival rate for the synfire propagation. In our simulations, we find that the synfire propagation can be divided into 
three types: the failed synfire propagation, the stable synfire propagation, as well as the synfire instability propagation 
(for detail, see Sec. 3.1). For neurons in each layer, a threshold method is developed to detect the local highest "energy" 
region. To this end, we use a 5 ms moving time window with 0.1 ms sliding step to count the number of spikes within each 
window. Here a high energy region means that the number of spikes within the window is larger than a threshold 6 — 50. 
Since we use a moving time window with small sliding step, there might be a continuous series of windows contain more 
than 50 spikes around a group of synchronous spikes. In this work, we only select the first window which covers the largest 
number of spikes around a group of synchronous spikes as the local highest energy region. We use the number of local 
highest energy region to determine which type of synfire propagation occurs. If there is no local highest energy region 
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detected in the final layer of the network, we consider it as the failed synfire propagation. When two or more separated 
local highest energy regions arc detected in one layer, we consider it as the synfire instability propagation. Otherwise, it 
means the occurrence of the stable synfire propagation. For each experimental setting, we carry out the simulation many 
times. The survival rate of the synfire propagation is defined as the ratio of the number of occurrence of the stable synfire 
propagation to the total number of simulations. In additional simulations, it turns out that the threshold value 9 can vary 
in a wide range without altering the results. Under certain conditions, noise can help the feedforward neuronal network 
produce the spontaneous spike packets, which promotes the occurrence of synfire instability propagation and therefore 
decreases the survival rate. For stable synfire propagation, there exists only one highest energy region for neurons in each 
layer. Spikes within this region are considered as the candidate synfire packet, which might also contain a few spontaneous 
spikes caused by noise and other factors. In this work, an adaptive algorithm is introduced to eliminate spontaneous spikes 
from the candidate synfire packet. Suppose now that there is a candidate synfire packet in the i-th layer with the number 
of spikes it contains and the corresponding spiking times {^1,^2, • ■ • i^ai}- The average spiking time of the candidate 
synfire packet is therefore given by 



We remove the j-th spike from the candidate synfire packet if it satisfies: \tj — ti\ > fiai, where is a parameter of our 
algorithm. Wc recompute the average spiking time as well as the standard deviation of the spiking times for the new 
candidate synfire packet, and repeat the above eliminating process, until no spike is removed from the new candidate 
synfire packet anymore. We define the remaining spikes as the synfire packet, which is characterized by the final values 
of ai and cr,;. Parameter /i determines the performance of the proposed algorithm. If fi is too large, the synfire packet 
will lose several useful spikes at its borders, and if fi is too small, the synfire packet will contain some noise data. In our 
simulations, we found that /i = 4 can result in a good compromise between these two extremes. It should be emphasized 
that our algorithm is inspired by the method given in (Gewaltig et al. 2001). Next, we introduce how to measure the 
performance of the firing rate propagation. The performance of firing rate propagation is evaluated by combining it with 
a population code. Specifically, we compute how similar the population firing rates in different layers to the external 
input current I{t) (van Rossum et al. 2002; Vogels and Abbott 2005). To do this, a 5 ms moving time window with 1 ms 
sliding step is also used to estimate the population firing rates (t) for different layers as well as the smooth version of the 
external input current Is{t). The correlation coefficient between the population firing rate of the i-th layer and external 




(2.9) 



Thus the standard deviation of the spiking times in the i-th layer can be calculated as follows: 




(2.10) 
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input current is calculated by 

^^^^ ^ ([/.(fc + r)-/.] [n{k)-u])^ 

y([/.(fc + r)-L]')^([r,(A:)-r,]2)^' ^^'^^^ 

where denotes the average over time. Here we use the maximum cross-correlation coefficient Qi ~ max{Ci(T)} to 
quantify the performance of the firing rate propagation in the i-th layer. Note that Qi is a normalization measure and a 
larger value corresponds to a better performance. 

2.5 Numerical simulation method 

In all numerical simulations, we use the standard Eulcr-Maruyama integration scheme to numerically calculate the afore- 
mentioned stochastic differential Eqs. (2.1)-(2.8) (Kloeden et al. 1994). The temporal resolution of integration is fixed 
at 0.02 ms for calculating the measures of the synfire mode and at 0.05 ms for calculating the measures of the firing 
rate mode, as the measurement of the synfire needs higher precise. In additional simulations, we have found that further 
reducing the integration time step does not change our numerical results in a significant way. For the synfire mode, all 
simulations are executed at least 100 ms to ensure that the synfire packet can be successfully propagated to the final 
layer of the considered network. While studying the firing rate mode, we perform all simulations up to 5000 ms to collect 
enough spikes for statistical analysis. It should be noted that, to obtain convincing results, we carry out several times of 
simulations (at least 200 times for the synfire mode and 50 times for the firing rate mode) for each experimental setting 
to compute the corresponding measures. 



3 Simulation results 

In this section, we report the main results obtained in the simulation. We first systematically investigate the signal 
propagation in the URE feedforward neuronal networks. Then, wc compare these results with those for the corresponding 
RRE feedforward neuronal networks. Finally, we further study the signal propagation in the UREI feedforward neuronal 
networks. 

3.1 Synfire propagation in URE feedforward neuronal networks 

Here we study the role of unreliable synapses on the propagation of synfire packet in the URE feedforward neuronal 
networks. In the absence of noise, we artificially let each sensory neuron fire and only fire an action potential at the 
same time {ai ~ 100 and cti = ms). Without loss of generality, we let all sensory neurons fire spikes at t = 10 ms. 
Figure 2 shows four typical spike raster diagrams of propagating synfire activity. Note that the time scales in Figs. 2(a)- 
2(d) are different. The URE feedforward neuronal network with both small successful transmission probability and small 
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excitatory synaptic strength badly supports the synfirc propagation. In this case, due to high synaptic unrehabihty and 
weak excitatory synaptic interaction between neurons, the propagation of synfirc packet cannot reach the final layer of the 
whole network (see Fig. 2(a)). For suitable values of p and g, we find that the synfirc packet can be stably transmitted 
in the URE feedforward neuronal network. Moreover, it is obvious that the width of the synfire packet at any layer for 
p ~ 0.8 is much narrower than that of the corresponding synfire packet for p = 0.25 (sec Figs. 2(b) and 2(c)). At the 
same time, the transmission speed is also enhanced with the increasing of p. These results indicate that the neuronal 
response of the considered network is much more precise and faster for suitable large successful transmission probability. 
However, our simulation results also reveal that a strong excitatory synaptic strength with large value of p might destroy 
the propagation of synfire activity. As we see from Fig. 2(d), the initial tight synfire packet splits into several different 
synfire packets during the transmission process. Such phenomenon is called the "synfire instability" (Tetzlaff et al. 2002; 
Tetzlaff et al. 2003), which mainly results from the burst firings of several neurons caused by the strong excitatory synaptic 
interaction as well as the stochastic fluctuation of the synaptic connections. 

In Fig. 3(a), we depict the survival rate of synfire propagation as a function of the successful transmission probability 
p for different values of excitatory synaptic strength g, with the noise intensity Dt ~ 0. We find that each survival 
rate curve can be at least characterized by one corresponding critical probability pon- For small p, due to low synaptic 
reliability, any synfire packet cannot reach the final layer of the URE feedforward neuronal network. Once the successful 
transmission probability p exceeds the critical probability pon, the survival rate rapidly transits from to 1, suggesting 
that the propagation of synfire activity becomes stable for a suitable high synaptic reliability. On the other hand, besides 
the critical probability pon, we find that the survival rate curve should be also characterized by another critical probability 
Poff if the excitatory synaptic strength is sufficiently strong (for example, g ~ 3.5 nS in Fig. 3(a)). In this case, when 
p > Poff, our simulation results show that the survival rate rapidly decays from 1 to 0, indicating that the network fails 
to propagate the stable synfire packet again. However, it should be noted that this does not mean that the synfire packet 
cannot reach the final layer of the network in this situation, but because the excitatory synapses with both high reliability 
and strong strength lead to the occurrence of the redundant synfire instability in transmission layers. 

To systematically establish the limits for the appearance of stable synfire propagation as well as to check that whether 
our previous results can be generalized within a certain range of parameters, we further calculate the survival rate of 
synfire propagation in the {g,p) panel, which is shown in Fig. 3(b). As we see, the whole {g,p) panel can be clearly 
divided into three regimes. These regimes include the failed synfire propagation regime (regime I), the stable synfire 
propagation regime (regime II), and the synfire instability propagation regime (regime HI). Our simulation results reveal 
that transitions between these regimes are normally very fast and therefore can be described as a sharp transition. The data 
shown in Fig. 3(b) further demonstrate that synfire propagation is determined by the combination of both the successful 
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Figure 2: (Color online) Several typical spike raster diagrams for different values of successful transmission probability 
p and excitatory synaptic strength g. Shown are samples of (a) failed synfire propagation, (b) and (c) stable synfire 
propagation, and (d) synfire instability. System parameters are p = 0.19 and g = 1 nS (a), p = 0.25 and 5 = 1 nS (b), 
p = 0.8 and g = 1 nS (c), and p ~ 0.7 and g = 2 nS (d), respectively. As we see, the synfire packet can reach the final 
layer of the network successfully only for appropriate values of p and g. It should be noted that the time scales in these 
four subfigures are different. 

transmission probability and excitatory synaptic strength. For a lower synaptic reliability, the URE feedforward neuronal 
network might need a larger g to support the stable propagation of synfire packet. 

In reality, not only the survival rate of the synfire propagation but also its performance is largely influenced by the 
successful transmission probability and the strength of the excitatory synapses. In Figs. 3(c) and 3(d), we present the 
standard deviation of the spiking times of the output synfire packet ctio for different values of p and g, respectively. Note 
that here we only consider parameters p and g within the stable synfire propagation regime. The results illustrated in 
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Figure 3: (Color online) Effects of the successful transmission probability and excitatory synaptic strength on the synfire 
propagation in URE feedforward neuronal networks, (a) The survival rate of the synfire propagation versus p for different 
values of g. (b) Schematic of three different synfire propagation regimes in the {g,p) panel (41 x 41 = 1681 points). 
Regime I: the failed synfire propagation region; regime II: the stable synfire propagation region; and regime III: the synfire 
instability propagation region, (c) The value of aio as a function oi p for different values of g. (d) The value of aio as a 
function of g for different values of p. In all cases, the noise intensity Dt = 0. Each data point shown here is computed 
based on 200 independent simulations with different random seeds. 

Fig. 3(c) clearly demonstrate that the propagation of synfire packet shows a better performance for a suitable higher 
synaptic reliability. For the ideal case p = I, the URE feedforward neuronal network even has the capability to propagate 
the perfect synfire packet (a^ = 100 and at — ms) in the absence of noise. On the other hand, it is also found that for 
a fixed p the performance of synfire propagation becomes better and better as the value of g is increased (see Fig. 3(d)). 
The above results indicate that both high synaptic reliability and strong excitatory synaptic strength are able to help the 
URE feedforward neuronal network maintain the precision of neuronal response in the stable synfire propagation regime. 
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Figure 4: (Color online) Dependence of the synfire propagation on the parameters of the initial spike packet. Here 
we display the survival rate of the synfire propagation versus ai (a) and cti (b) for different successful transmission 
probabilities, and the values of n,; (c) and ai (d) as a function of the layer number for different initial spike packets and 
different intrinsic parameters of the network, respectively. Note that the vertical axis in (d) is a log scale. In all cases, the 
noise intensity Dt = 0. Other parameters are g = 1 nS and <ti = 3 ms (a), g = 1 nS and ai = 70 (b). Each data shown 
in (a) and (b) is computed based on 600 independent simulations, whereas each data shown in (c) and (d) is calculated 
based on 500 independent successful synfire propagation simulations. 

Up to now, we only use the perfect initial spike packet (ai ~ 100 and cri = ms) to evoke the synfire propagation. 
This is a special case which is simplified for analysis, but it is not necessary to restrict this condition. To understand 
how a generalized spike packet is propagated through the URE feedforward neuronal network, we randomly choose ai 
neurons from the sensory layer, and let each of these neurons fire and only fire a spike at any moment according to a 
Gaussian distribution with the standard deviation cti. In Figs. 4(a) and 4(b), we plot the survival rate of the synfire 
propagation as a function of ai and <7i for four different values of successful transmission probability, respectively. When 
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the successful transmission probability is not too large (for example, p ~ 0.24, 0.27, and 0.3 in Figs. 4(a) and 4(b)), the 
synfirc activity is well build up after several initial layers for sufficiently strong initial spike packet (large ai and small cti), 
and then this activity can be successfully transmitted along the entire network with high survival rate. In this case, too 
weak initial spike packet (small ai and large cri) leads to the propagation of the neural activities becoming weaker and 
weaker with the increasing of layer number. Finally, the neural activities are stopped before they reach the final layer of 
the network. Moreover, with the increasing of the successful transmission probability, neurons in the downstream layers 
will share more common synaptic currents from neurons in the corresponding upstream layers. This means that neurons 
in the considered network have the tendency to fire more synchronously for suitable larger p (not too large). On the 
other hand; for sufficiently high synaptic reliability (for instance, p = 0.6 in Figs. 4(a) and 4(b)), a large ai or a suitable 
large tri may result in the occurrence of synfire instability, which also reduces the survival rate of the synfire propagation. 
Therefore, for a fixed g, the URE feedforward neuronal network with suitable higher synaptic reliability has the ability to 
build up stable synfire propagation from a slightly weaker initial spike packet (see Figs. 4(a) and 4(b)). 

Figures 4(c) and 4(d) illustrate the values of ai and ai versus the layer number for different initial spike packets 
and several different intrinsic system parameters of the network (the successful transmission probability p and excitatory 
synaptic strength g). For each ease shown in Figs. 4(c) and 4(d), once the synfire propagation is successfully established, 
rii converges fast to the saturated value 100 and ct, approaches to an asymptotic value. Although the initial spike packet 
indeed determines whether the synfire propagation can be established or not as well as influences the performance of 
synfire propagation in the first several layers, but it does not determine the value of <7i in deep layers provided that the 
synfire propagation is successfully evoked. For the same intrinsic system parameters, if we use different initial spike packets 
to evoke the synfire propagation, the value of Ui in deep layers is almost the same for different initial spike packets (see 
Fig. 4(d)). The above results indicate that the performance of synfire propagation in deep layers of the URE feedforward 
neuronal network is quite stubborn, which is mainly determined by the intrinsic parameters of the network but not the 
parameters of the initial spike packet. In fact, many studies have revealed that the synfire activity is governed by a stable 
attractor in the [a, a) space (Diesmann et al. 1999; Diesmann et al. 2001; Diesmann 2002; Gewaltig et al. 2001). Our 
above finding is a signature that the stable attractor of synfire propagation does also exist for the feedforward neuronal 
networks with unreliable synapses. 

Next, we study the dependence of synfire propagation on neuronal noise. It is found that both the survival rate of 
synfire propagation and its performance are largely influenced by the noise intensity. There is no signiflcant qualitative 
difference between the corresponding survival rate curves in low synaptic reliable regime. However, we flnd important 
differences between these curves for small g in high synaptic reliable regime, as well as for large g in intermediate synaptic 
reliable regime, that is, during the transition from the successful synfire regime to the synfire instability propagation 
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Figure 5: (Color online) Effects of the noise intensity D on the synfire propagation, (a) The survival rate versus successful 
transmission probability p for different excitatory coupling strength and noise intensities, (b) The value of aio versus the 
noise intensity D for different successful transmission probabilities, with the excitatory synaptic strength g = 1.5 nS. In 
all cases, the parameters of the initial spike packet arc ai = 100 and ai = ms. Each data shown here is computed based 
on 200 independent simulations with different random seeds. 

regime (see from Fig. 5(a)). For each case, it is obvious that the top region of the survival rate becomes smaller with 
the increasing of noise intensity. This is at least due to the following two reasons: (i) noise makes neurons desynchronize, 
thus leading to a more dispersed synfire packet in each layer. For relatively high synaptic reliability, a dispersed synfire 
packet has the tendency to increase the occurrence rate of the synfire instability, (ii) Noise with large enough intensity 
results in several spontaneous neural firing activities at random moments, which also promote the occurrence of the synfire 
instability. Figure 5(b) presents the value of (Tiq as a function of the noise intensity Dt for different values of successful 
transmission probability p. As we see, the value of ctiq becomes larger and larger as the noise intensity is increased from 
to 0.1 (weak noise regime). This is also due to the fact that the existence of noise makes neurons desynchronize in 
each layer. However, although noise tends to reduce the synchrony of synfire packet, the variability of ai in deep layers is 
quite low (data not shown). The results suggest that, in weak noise regime, the synfire packet can be stably transmitted 
through the feedforward neuronal network with small fiuctuation in deep layers, but displays slightly worse performance 
compared to the case oi Dt = 0. Further increase of noise will cause many spontaneous neural firing activities which 
might significantly deteriorate the performance of synfire propagation. However, it should be emphasized that, although 
the temporal spread of synfire packet tends to increase as the noise intensity grows, several studies have suggested that 
under certain conditions the basin of attraction of synfire activity reaches a maximum extent (Diesmann 2002; Postma 
et al. 1996; Boven and Acrtscn 1990). Such positive effect of noise can be compared to a well known phenomenon called 
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Figure 6: The maximum eross-eorrelation coefficient between the smooth version of external input current Isit) and the 
population firing rate of sensory neurons ri (t) for different noise intensities. 

aperiodic stochastic resonance (Collins et al. 1995b; Collins et al. 1996; Diesmann 2002). 
3.2 Firing rate propagation in URE feedforward neuronal networks 

In this subsection, we examine the firing rate propagation in URE feedforward neuronal networks. To this end. we assume 
that all sensory neurons are injected to a same time- varying external current I{t) (see Sec. 2.2 for detail). Note now that 
the sensory neurons are modeled by using the integrate-and-fire neuron model in the study of the firing rate propagation. 

Before we present the results of the firing rate propagation, let us first investigate how noise infiuences the encoding 
capability of sensory neurons by the population firing rate. This is an important preliminary step, because how much 
input information represented by sensory neurons will directly infiuence the performance of firing rate propagation. The 
corresponding results are plotted in Figs. 6 and 7, respectively. When the noise is too weak, the dynamics of sensory 
neurons is mainly controlled by the same external input current, which causes neurons to fire spikes almost at the same 
time (see Figs. 7(a) and 7(b)). In this case, the information of the external input current is poorly encoded by the 
population firing rate since the synchronous neurons have the tendency to redundantly encode the same aspect of the 
external input signal. When the noise intensity falls within a special intermediate range (about 0.5-10), neuronal firing is 
driven by both the external input current and noise. With the help of noise, the firing rate is able to refiect the temporal 
structural information (i.e., temporal waveform) of the external input current to a certain degree (see Figs. 7(c) to 7(f)), 
and therefore Qi has large value in this situation. For too large noise intensity, the external input current is almost 
drowned in noise, thus resulting that the input information cannot be well read from the population firing rate of sensory 
neurons again. On the other hand, sensory neurons can fire "false" spikes provided that they are driven by sufficiently 
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Figure 7: (Color online) Impacts of noise on the encoding performance (the firing rate mode) of sensory neurons. For each 
case, the smooth version of the external input current Igit) (top panel), spike raster diagram of sensory neurons (middle 
panel), and population firing rate of sensory neurons (bottom panel) are shown. Noise intensities are Dg ~ (a), Ds =0.1 
(b), Ds = 0.6 (c), Ds ^ 0.8 (d), = 2 (c), and Ds^5 (f), respectively. 

strong noise (as for example at t ^ 2800 ms in Fig. 7(c)). Although the encoding performance of the sensory neurons might 
be good enough in this case, our numerical simulations reveal that such false spikes will seriously reduce the performance 
of the firing rate propagation in deep layers, which will be discussed in detail in the later part of this section. By taking 
these factors into account, we consider the noise intensity of sensory neurons to be within the range of 0.5 to 1 in the 
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Figure 8: An example of the firing rate propagation in the URE feedforward network. Here we show the smooth version 
of the external input current Is{t), as and the population firing rates of layers 1, 2, 4, 6, 8, and 10, respectively. System 
parameters are g = 0.4 nS, p = 0.2, and Dg = Dt ~ 0.7. 

present work. 

Figure 8 shows a typical example of the firing rate propagation. In view of the overall situation, the firing rate can be 
propagated rapidly and basically linearly in the URE feedforward neuronal network. However, it should be noted that, 
although the firing rates of neurons from the downstream layers tend to track those from the upstream layers, there are 
still several differences between the firing rates for neurons in two adjacent layers. For example, it is obvious that some 
low firing rates may disappear or be slightly amplified in the first several layers, as well as some high firing rates are 
weakened to a certain degree during the whole transmission process. Therefore, as the neural activities are propagated 
across the network, the firing rate has the tendency to lose a part of local detailed neural information but can maintain 
a certain amount of global neural information. As a result, the maximum cross-correlation coefficient between /s(t) and 
ri{t) basically drops with the increasing of the layer number. 

Let us now assess the impacts of the unreliable synapses on the performance of firing rate propagation in the URE 
feedforward neuronal network. Figure 9(a) presents the value of Qio versus the success transmission probability p for 
various excitatory synaptic strengths. For a fixed value of f/, a bell-shaped Qio curve is clearly seen by changing the 
value of successful transmission probability, indicating that the firing rate propagation shows the best performance at an 
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Figure 9: (Color online) Effects of the unreliable synapses on the performance of firing rate propagation, (a) The value 
of Qio as a function oi p for different values of excitatory synaptic strength, (b) The value of Qio as a function of g for 
different values of successful transmission probability. Noise intensities are Dg ^ Dt ~ 0.5 in all cases. Here each data 
point is computed based on 50 different independent simulations with different random seeds. 

optimal synaptic reliability level. This is because, for each value of g, a very small p will result in the insufficient firing 
rate propagation due to low synaptic reliability, whereas a sufficiently large p can lead to the excessive propagation of 
firing rate caused by burst firings. Based on above reasons, the firing rate can be well transmitted to the final layer of the 
URE feedforward neuronal network only for suitable intermediate successful transmission probabilities. Moreover, with 
the increasing of the considered network needs a relatively small p to support the optimal firing rate propagation. In 
Fig. 9(b), we plot the value of Qio as a function of the excitatory synaptic strength g for different values oi p. Here the 
similar results as those shown in Fig. 9(a) can be observed. This is due to the fact that increasing g and fixing the value 
of p is equivalent to increasing p and fixing the value of g to a certain degree. According to the aforementioned results, 
we conclude that both the successful transmission probability and excitatory synaptic strength are critical for firing rate 
propagation in URE feedforward networks, and better choosing of these two unreliable synaptic parameters can help the 
cortical neurons encode neural information more accurately. 

Next, we examine the dependence of the firing rate propagation on neuronal noise. The corresponding results are 
plotted in Figs. 10 and 11, respectively. Figure 10(a) demonstrates that the noise of cortical neurons plays an important 
role in firing rate propagation. Noise of cortical neurons with appropriate intensity is able to enhance their encoding 
accuracy. It is because appropriate intermediate noise, on the one hand, prohibits synchronous firings of cortical neurons 
in deep layers, and on the other hand, ensures that the useful neural information does not drown in noise. However, 
the level of enhancement is largely influenced by the noise intensity of sensory neurons. As we sec, for a large value of 
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Figure 10: (Color online) Impacts of noise on the performance of firing rate propagation, (a) The value of Qio as a function 
of the noise intensity of cortical neurons Dt for different values of Dg. (b) The performance of firing rate propagation in 
each layer at Dt ~ 0.6 for two different noise intensities of sensory neurons. In all cases, p — 0.2 and g ~ 0.4 nS. Here 
each data point is computed based on 50 different independent simulations with different random seeds. 

such enhancement is weakened to a great extent. This is because slightly strong noise intensity of sensory neurons 
will cause these neurons to fire several false spikes and a part of these spikes can be propagated to the transmission 
layers. If enough false spikes appear around the weak components of the external input current, these spikes will help 
the network abnormally amplify these weak components during the whole transmission process. The aforementioned 
process can be seen clearly from an example shown in Fig. 11. As a result, the performance of the firing rate propagation 
might be seriously deteriorated in deep layers. However, it should be noted that this kind of influence typically needs the 
accumulation of several layers. Our simulation results show that the performance of firing rate propagation can be well 
maintained or even becomes slightly better (depending on the noise intensity of sensory neurons, sec Fig. 6) in the first 
several layers for large Dg (see Fig. 10(b)). In fact, the above results are based on the assumption that each cortical neuron 
is driven by independent noise current with the same intensity. Our results can be generalized from the sensory layer to 
the transmission layers if we suppose that noise intensities for neurons in different transmission layers are different. All 
these results imply that better tuning of the noise intensities of both the sensory and cortical neurons can enhance the 
performance of firing rate propagation in the URE feedforward neuronal network. 

3.3 Stochastic effect of neurotransmitter release 

From the numerical results depicted in Sees. 3.1 and 3.2, we find that increasing g with p fixed has similar effects as 
increasing p while keeping g fixed for both the synfire mode and firing rate mode. Some persons might therefore postulate 
that the signal propagation dynamics in feedforward neuronal networks with unreliable synapses can be simply determined 

20 



4 




2500 3000 3500 4000 

time (ms) 



Figure 11: (Color online) An example of weak external input signal amplification. System parameters are the suceessful 
transmission probability p = 0.2, excitatory synaptic strength g ~ 0.4 nS, and noise intensities Ds ~ 2.5 and Df ~ 0.6, 
respectively. 

by the average amount of received neurotransmitter for each neuron in a time instant, which can be refiected by the product 
of g ■ p. To check whether this is true, we calculate the measures of these two signal propagation modes as a function of 
g ■ p for different successful transmission probabilities. If this postulate is true, the URE feedforward neuronal network 
will show the same propagation performance for different values of p at a fixed g - p. Our results shown in Figs. 12(a)-12(c) 
clearly demonstrate that the signal propagation dynamics in the considered network can not be simply determined by 
the product g ■ p oi, equivalently, by the average amount of received neurotransmitter for each neuron in a time instant. 
For both the synfire propagation and firing rate propagation, although the propagation performance exhibits the similar 
trend with the increasing oi g ■ p, the corresponding measure curves do not superpose in most parameter region for each 
case, and in some parameter region the differences are somewhat significant (see Figs. 12(b) and 12(c)). This is because of 
the stochastic effect of neurotransmitter release, that is, the unreliability of neurotransmitter release will add randomness 
to the system. Different successful transmission probabilities may introduce different levels of randomness, which will 
further affect the nonlinear spiking dynamics of neurons. Therefore, the URE feedforward neuronal network might display 
different propagation performance for different values of p even at a fixed g ■ p. If we set the value oi g ■ p constant, a low 
synaptic reliability will introduce large fluctuations in the synaptic inputs. For small p, according to the above reason, 
some neurons will fire spikes more than once in the large g -p regime. This mechanism increases the occurrence rate of the 
synfire instability. Thus, the URE feedforward neuronal network has the tendency to stop the stable synfire propagation 
for a small synaptic transmission probability (see Fig. 12(a)). On the other hand, a high synaptic reliability will introduce 
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Figure 12: (Color online) Dependence of signal propagation dynamics on the product oi g ■ p in the URE feedforward 
neuronal network. Synfire mode: survival rate (a) and trio versus g ■ p with Dt = 0. The parameters of the initial spike 
packet are ai = 100 and ai ~ ms. Firing rate mode: Qio as a function oi g ■ p with Dg = Dt = 0.5. Here each data 
point shown in (a) and (b) is calculated based on 200 different independent simulations, whereas each data point shown 
in (c) are based on 50 different independent simulations. 

small fluctuations in the synaptic inputs for a fixed g ■ p. This makes neurons in the considered network fire spikes almost 
synchronously for a large p, thus resulting the worse performance for the firing rate propagation in large g ■ p regime (see 
Fig. 12(c)). Our above results suggest that the performance of the signal propagation in feedforward neuronal networks 
with unreliable synapses is not only purely determined by the change of synaptic parameters, but also largely influenced 
by the stochastic effect of neurotransmitter release. 

3.4 Comparison with corresponding RRE feedforward neuronal networks 

In this subsection, we make comparisons on the propagation dynamics between the URE and the RRE feedforward 
networks. We first introduce how to generate a corresponding RRE feedforward neuronal network for a given URE 
feedforward neuronal network. Suppose now that there is a URE feedforward neuronal network with successful transmission 
probability p. A corresponding RRE feedforward neuronal network is constructed by using the connection density p (on 
the whole), that is, a synapse from one neuron in the upstream layer to one neuron in the corresponding downstream layer 
exists with probability p. As in the URE feedforward neuronal network given in Sec. 2.1, there is no feedback connection 
from downstream neurons to upstream neurons and also no connection among neurons within the same layer in the RRE 
feedforward neuronal network. It is obvious that parameter p has different meanings in these two different feedforward 
neuronal network models. The synaptic interactions between neurons in the RRE feedforward neuronal network are also 
implemented by using the conductance-based model (see Eqs. (2.6) and (2.7) for detail). However, here we remove the 
constraint of the synaptic reliability parameter for the RRE feedforward neuronal network, e.g., h{i,j;k,j — 1) = 1 in 
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Figure 13: (Color online) The dilFerence between the synfire propagation in the URE feedforward neuronal network and 
the RRE feedforward neuronal network. Here we show the value of survival rate as a function of ai for different network 
models. In all cases, Dt ^ and qi = 70. Other system parameters are 5 = 4 nS and p = 0.15 (dot: and circle: "o" 
), and g = 3.5 nS and p ~ 0.12 (square: and asterisk: "*"). Each data point is calculated based on 500 different 

independent simulations with different random seeds. 

all cases. A naturally arising question is what are the differences, if have, between the synfire propagation and firing 
propagation in URE feedforward neuronal networks and those in RRE feedforward neuronal networks, although the 
numbers of active synaptic connections that taking part in transmitting spikes in a time instant are the same from the 
viewpoint of mathematical expectation. 

For the synfire propagation, our simulation results indicate that, compared to the RRE feedforward neuronal network, 
the URE feedforward neuronal network is able to suppress the occurrence of synfire instability to a certain degree, which 
can be seen clearly in Fig. 13. Typically, this phenomenon can be observed in strong excitatory synaptic strength regime. 
Due to the heterogeneity of connectivity, some neurons in the RRE feedforward neuronal network will have more input 
synaptic connections than the other neurons in the same network. For large value of g, these neurons tend to fire spikes 
very rapidly after they received synaptic currents. If the width of the initial spike packet is large enough, these neurons 
might fire spikes again after their refractory periods, which are induced by a few spikes from the posterior part of the 
dispersed initial spike packet. These spikes may increase the occurrence rate of the synfire instability. While in the case 
of URE feedforward neuronal network, the averaging effect of unreliable synapses tends to prohibit neurons fire spikes 
too quickly. Therefore, under the equivalent parameter conditions, less neurons can fire two or more spikes in the URE 
fcedforeard neuronal network. As a result, the survival rate of the synfire propagation for the URE fcedforeard neuronal 
network is larger than that for the RRE feedforward neuronal network (see Fig. 13), though not so significant. 
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Figure 14: (Color online) Firing rate propagation in URE feedforward neuronal network and RRE feedforward neuronal 
network. The value of Qiq as a function of excitatory synaptic strength g for p = 0.2 (a) and p = 0.6 (b), respectively. 
Noise intensities are Dt = Dg = 0.5. Each data point is calculated based on 50 different independent simulations with 
different random seeds. 

In further simulations, we find interesting results in small p regime for the firing rate propagation. Compared to the 
case of the URE feedforward neuronal network, the RRE feedforward neuronal network can better support the firing 
rate propagation in this small p regime for strong excitatory synaptic strength (see Fig. 14(a)). It is because the long- 
time averaging effect of unreliable synapses at small p tends to make neurons fire more synchronous spikes in the URE 
feedforward neuronal network through the homogcnization process of synaptic currents. However, with the increasing 
of p, neurons in the downstream layers have the tendency to share more common synaptic currents from neurons in the 
corresponding upstream layers for both types of feedforward neuronal networks. The aforementioned factor makes the 
difference of the performance of firing rate propagation between these two types of feedforward neuronal networks become 
small so that the Qio curves almost coincide with each other for the case of p = 0.6 (see Fig. 14(b)). 

Although from the above results we can not conclude that unreliable synapses have advantages and play specific 
functional roles in signal propagation, not like those results shown in the previous studies (Goldman et al. 2002; Goldman 
2004), at least it is shown that the signal propagation activities are different in URE and RRE to certain degrees. We 
should be cautioned when using random connections to replace unreliable synapses in modelling research. However, it 
should be noted that the RRE feedforward neuronal network considered here is just one type of diluted feedforward 
neuronal networks. There exists several other possibilities to construct the corresponding diluted feedforward neuronal 
networks (Hehl et al. 2001). The similar treatments for these types of diluted feedforward neuronal networks require 
further investigation. 
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3.5 Signal propagation in UREI feedforward neuronal networks 

In this subsection, we further study the signal propagation in the feedforward neuronal networks composed of both exci- 
tatory and inhibitory neurons connected in an all-to-all coupling fashion (i.e., the UREI feedforward neuronal networks). 
This study is necessary because real biological neuronal networks, especially mammalian neocortex, consist not only of 
excitatory neurons but also of inhibitory neurons. The UREI feedforward neuronal network studied in this subsection has 
the same topology as that shown in Fig. 1. In simulations, we randomly choose 80 neurons in each layer as excitatory and 
the rest of them as inhibitory, as the ratio of excitatory to inhibitory neurons is about 4 : 1 in mammalian neocortex. The 
dynamics of the unreliable inhibitory synapse is also modeled by using Eqs. (2.6) and (2.7). The reversal potential of the 
inhibitory synapse is fixed at -75 mV, and its strength is set as J = A' • g, where _ftr is a scale factor used to control the 
relative strength of inhibitory and excitatory synapses. Since the results of the signal propagation in UREI feedforward 
neuronal networks are quite similar to those in URE feedforward neuronal networks, we omit most of them and only 
discuss the effects of inhibition in detail. 

Figure 15 shows the survival rate of synfire propagation in the {K,p) panel for three different excitatory synaptic 
strengths. Depending on whether the synfire packet can be successfully and stably transmitted to the final layer of the 
UREI feedforward neuronal network, the whole {K,p) panel can also be divided into three regimes. For each considered 
case, the network with both small successful transmission probability and strong relative strength of inhibitory and 
excitatory synapses (failed synfire regime) prohibits the stable propagation of the synfire activity. While in the case of 
high synaptic reliability and small K (synfire instability propagation regime), the synfire packet also cannot be stably 
transmitted across the whole network due to the occurrence of synfire instability. Therefore, the UREI feedforward 
neuronal network is able to propagate the synfire activity successfully in a stable way only for suitable combination of 
parameters p and K . Moreover, due to the competition between excitation and inhibition, the transitions between these 
different regimes cannot be described as a sharp transition anymore, in particular, for large scale factor K. Our results 
suggest that such non-sharp character is strengthen with the increasing of g. On the other hand, the partition of these 
different propagation regimes depends not only on parameters p and K but also on the excitatory synaptic strength g. 
As the value of g is decreased, both the synfire instability propagation regime and stable synfire propagation regime are 
shifted to the upper left of the {K,p) panel at first, and then disappear one by one (data not shown). In contrast, a 
strong excitatory synaptic strength has the tendency to extend the areas of the synfire instability propagation regime, and 
meanwhile makes the stable synfire propagation regime move to the lower right of the {K,p) panel. 

For the case of firing rate propagation, we plot the value of Qio versus the scale factor K for different excitatory 
synaptic strengths in Fig. 16, with a fixed successful transmission probability p = 0.2. When the excitatory synaptic 
strength is small (for instance g — 0.4 nS), due to weak excitatory synaptic interaction between neurons the UREI 
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Figure 15: (Color online) Partition of three different synfire propagation regimes in the {K,p) panel (41 x 41 = 1681 
points). Regime I: the failed synfire propagation region; regime II: the stable synfire propagation region; and regime III: 
the synfire instability propagation region, we set g ~ 2.5 nS (a), .9 = 3 nS (b), and g ~ 3.5 nS (c). In all cases, the 
parameters of the initial spike packet are ai = 100 and ai = ms, and the noise intensity is Dt — 0. Each data point 
shown here is calculated based on 200 different independent simulations with different random seeds. 

feedforward neuronal network cannot transmit the firing rate sufficiently even for K ~ 0. In this case, less and less neural 
information can be propagated to the final layer of the considered network with the increasing of K. Therefore, Qio 
monotonically decreases with the scale factor K at first and finally approaches to a low steady state value. Note that here 
the low steady state value is purely induced by the spontaneous neural firing activities, which are caused by the additive 
Gaussian white noise. As the excitatory synaptic strength grows, more neural information can be successfully transmitted 
for small value of K. When g is increased to a rather large value, such as g — 0.6 nS, the coupling is so strong that too 
small scale factor will lead to the excessive propagation of firing rate. However, in this case, the propagation of firing 
rate can still be suppressed provided that the relative strength of inhibitory and excitatory synapses is strong enough. As 
a result, there always exists an optimal scale factor to best support the firing rate propagation for each large excitatory 
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Figure 16: (Color online) Effect of inhibition on firing rate propagation. Here we show the value of Qiq as a function of 
scale factor K for different excitatory synaptic strengths. System parameters are p = 0.2, and Dg = Dt = 0.6 in all cases. 
Each data point is calculated based on 50 different independent simulations with different random seeds. 

synaptic strength (see Fig. 16). If we fix the value of g (not too small), then the similar results can also be observed by 
changing the scale factor for a large successful transmission probability (data not shown). Once again, this is due to the 
fact that increasing g and fixing p is equivalent to increasing p and fixing g to a certain degree. 

4 Conclusion and discussion 

The feedforward neuronal network provides us an effective way to examine the neural activity propagation through 
multiple brain regions. Although biological experiments suggested that the communication between neurons is more or 
less unreliable (Abeles 1991; Raastad et al. 1992; Smctters and Zador 1996), so far most relevant computational studies 
only considered that neurons transmit spikes based on reliable synaptic models. In contrast to these previous work, 
we took a different approach in this work. Here we first built a 10-laycr feedforward neuronal network by using purely 
excitatory neurons, which are connected with unreliable synapses in an all-to-all coupling fashion, that is, the so-called 
URE feedforward neuronal network in this paper. The goal of this work was to explore the dependence of both the synfire 
propagation and firing rate propagation on unreliable synapses in the URE neuronal network, but was not limited this 
type of feedforward neuronal network. Our modelling methodology was motivated by experimental results showing the 
probabilistic transmitter release of biological synapses (Branco and Staras 2009; Katz 1966; Katz 1969; Trommershauser 
et al. 1999). 

In the study of synfire mode, it was observed that the synfire propagation can be divided into three types (i.e., the 
failed synfire propagation, the stable synfire propagation, and the synfire instability propagation) depending on whether 




27 



the synfire packet can be successfully and stably transmitted to the final layer of the considered network. We found that 
the stable synfire propagation only occurs in the suitable region of system parameters (such as the successful transmission 
probability and excitatory synaptic strength). For system parameters within the stable synfire propagation regime, it 
was found that both high synaptic reliability and strong excitatory synaptic strength are able to support the synfire 
propagation in feedforward neuronal networks with better performance and faster transmission speed. Further simulation 
results indicated that the performance of synfire packet in deep layers is mainly influenced by the intrinsic parameters of 
the considered network but not the parameters of the initial spike packet, although the initial spike packet determines 
whether the synfire propagation can be evoked to a great degree. In fact, this is a signature that the synfire activity is 
governed by a stable attractor, which is in agreement with the results given in (Diesmann et al. 1999; Diesmann et al. 
2001; Diesmann 2002; Gewaltig et al. 2001). 

In the study of firing rate propagation, our simulation results demonstrated that both the successful transmission 
probability and the excitatory synaptic strength are critical for firing rate propagation. Too small successful transmission 
probability or too weak excitatory synaptic strength results in the insufficient firing rate propagation, whereas too large 
successful transmission probability or too strong excitatory synaptic strength has the tendency to lead to the excessive 
propagation of firing rate. Theoretically speaking, better tuning of these two synaptic parameters can help neurons encode 
the neural information more accurately. 

On the other hand, neuronal noise is ubiquitous in the brain. There are many examples confirmed that noise is able 
to induce many counterintuitive phenomena, such as stochastic resonance (Collins et al. 1995a; Collins et al. 1995b; 
Collins et al. 1996; Chialvo et al. 1997; Guo and Li 2009) and coherence resonance (Pikovsky and Kurths 1996; Lindner 
and Schimansky-Geier 2002; Guo and Li 2009). Here we also investigated how the noise influences the performance of 
signal propagation in URE feedforward neuronal networks. The numerical simulations revealed that noise tends to reduce 
the performance of synfire propagation because it makes neurons desynchronized and causes some spontaneous neural 
firing activities. Further studies demonstrated that the survival rate of synfire propagation is also largely infiuenced by 
the noise. In contrast to the synfire propagation, our simulation results showed that noise with appropriate intensity 
is able to enhance the performance of firing rate propagation in URE feedforward neuronal networks. In essence, it is 
because suitable noise can help neurons in each layer maintain more accurate temporal structural information of the the 
external input signal. Note that the relevant mechanisms about noise have also been discussed in several previous work 
(van Rossum et al. 2002; Masuda and Aihara 2002; Masuda and Aihara 2003), and our results are consistent with the 
findings given in these work. 

Furthermore, we have also investigated the stochastic effect of neurotransmitter release on the performance of signal 
propagation in the URE feedforward neuronal networks. For both the synfire propagation and firing rate propagation. 
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we found that the URE feedforward neuronal networks might display different propagation performance, even when their 
average amount of received neurotransmitter for each neuron in a time instant remains unchanged. This is because the 
unreliability of neurotransmitter release will add randomness to the system. Different synaptic transmission probabilities 
will introduce different levels of stochastic effect, and thus might lead to different spiking dynamics and propagation per- 
formance. These findings revealed that the signal propagation dynamics in feedforward neuronal networks with unreliable 
synapses is also largely influenced by the stochastic effect of neurotransmitter release. 

Finally, two supplemental work has been also performed in this paper. In the first work, we compared both the 
synfire propagation and firing rate propagation in URE feedforward neuronal networks with the results in corresponding 
feedforward neuronal networks composed of purely excitatory neurons but connected with reliable synapses in an random 
coupling fashion (RRE feedforward neuronal network). Our simulations showed that several different results exist for both 
the synfire propagation and firing rate propagation in these two different feedforward neuronal network models. These 
results tell us that we should be cautioned when using random connections to replace unreliable synapses in modelling 
research. In the second work, we extended our results to more generalized feedforward neuronal networks, which consist 
not only of the excitatory neurons but also of inhibitory neurons (UREI feedforward neuronal network) . The simulation 
results demonstrated that inhibition also plays an important role in both types of neural activity propagation, and better 
choosing of the relative strength of inhibitory and excitatory synapses can enhance the transmission capability of the 
considered network. 

The results presented in this work might be more realistic than those obtained based on reliable synaptic models. This 
is because the communication between biological neurons indeed displays the unreliable properties. In real neural systems, 
neurons may make full use of the characteristics of unreliable synapses to transmit neural information in an adaptive way, 
that is, switching between different signal propagation modes freely as required. Further work on this topic includes at least 
the following two aspects: (i) since all our results are derived from numerical simulations, an analytic description of the 
synfire propagation and firing rate propagation in our considered feedforward neuronal networks requires investigation, (ii) 
Intensive studies on signal propagation in the feedforward neuronal network with other types of connectivity, such as the 
Mexican-liat-type connectivity (Hamaguchi et al. 2004; Hamaguchi and Ailiara 2005) and the Gaussian-type connectivity 
(van Rossum et al. 2002), as well as in the feedforward neuronal network imbedded into a recurrent network (Aviel et al. 
2003; Vogels and Abbott 2005; Kumar et al. 2008), from the unreliable synapses point of view are needed as well. 
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